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MULTI-SERVER  QUEUES 


Sheldon  M.  Ross 

Department  of  Industrial  Engineering  and 
Operations  Research 


ABSTRACT 

X 

We  will  survey  a  variety  of  multiserver  models  in  which  the 
arrival  stream  is  a  Poisson  process.  In  particular,  we  will 
consider  the  Erlang  loss  model  in  which  arrivals  finding  all 
servers  busy  are  lost.  In  this  syscem,  we  assume  a  general 
service  distribution.  We  will  also  consider  finite  and  infinite 
capacity  versions  of  this  model.  Another  model  of  this  type  is 
the  shared  processor  system  in  which  service  is  shared  by  all 
customers . 

Another  model  to  be  considered  is  the  G/M/k  in  which  arrivals 
are  in  accordance  with  a  renewal  process  and  the  service 
distribution  is  exponential.  We  will  analyze  this  model  by 
means  of  the  embedded  Markov  chain  approach. 

f - — — -- 

0.  INTRODUCTION 

We  will  consider  some  multiserver  queueing  models.  In 
Section  1,  we  deal  with  the  Erlang  loss  model  which  supposes 
Poisson  arrivals  and  a  general  service  distribution  G  .  By  use 
of  a  "reversed  process"  argument  (see  [2])  we  will  indicate  a 
proof  of  the  well-known  result  chat  the  distribution  of  number 
of  busy  servers  depends  on  G  only  through  its  mean.  In 
Section  2  we  then  analyze  a  shared-processor  model  in  which  the 
servers  are  able  to  combine  forces.  Again  making  use  of  the 
reverse  process,  we  obtain  the  limiting  distribution  for  this 
model.  In  Section  3  we  review  the  embedded  Markov  chain  approach 
for  the  G/M/k  model;  and  in  the  final  section  we  present  the 
model  M/G/k. 


r 
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1.  THE  ERLANG  LOSS  SYSTEM 


One  of  the  most  basic  types  of  queueing  system  are  the  loss 
systems  in  which  an  arrival  that  finds  all  servers  busy  is  pre¬ 
sumed  lost  to  the  system.  The  simplest  such  system  is  the  M/M/k 
loss  system  in  which  customers  arrive  according  to  a  Poisson 
process  having  rate  X  ,  enter  the  system  if  at  least  one  of  the 
k  servers  is  free,  and  then  spend  an  exponential  amount  of  time 
with  rate  p  being  served.  The  balance  equations  for  the  sta¬ 
tionary  probabilities  are 


State 


Rate  leave  *  rate  enter 


0 

i  ,  0  <  i  <  k 
k 


XP0  -  UPX 

(x  +  ip)p.  -  (i  +  DuPi+1 

kwpk  *  xpk-l  • 


XP 


i-1 


Using  the  equation 
to  give 


1  ,  the  above  equations  can  be  solved 


_  (X/u) 1/i! 

i  k 

l  (X/u)J/j! 

j-0 


i  *  0,1,  ...,  k  . 


Since  E [ S J  *  1/u  ,  where  E [ S ]  is  the  mean  service  time,  the 
above  can  be  written  as 


P  .  immhn  . 

i  k 

l  (XE[S])J/j! 

j-0 


i  -  0,1 . k  . 


The  above  was  originally  obtained  by  Erlang  who  then  con¬ 
jectured  that  it  was  valid  for  an  arbitrary  service  distribution. 
We  shall  present  a  proof  of  this  result,  known  as  the  Erlang  loss 
formula  when  the  service  distribution  G  is  continuous  and  has 
density  g  . 


Theorem  1.  The  limiting  distribution  of  the  number  of 
customers  in  the  Erlang  loss  system  is  given  by 


Pin  in  „.t«>  -  .  „  .  o,l . „ 

I  (XEISI)1/!! 
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and  given  chat  chere  are  n  in  the  system,  the  ages  (or  the  re¬ 
sidual  times)  of  these  n  are  independent  and  identically  dis¬ 
tributed  according  to  the  equilibrium  distribution  of  G  . 

Proof .  We  can  analyze  the  above  system  as  a  Markov  process 
by  letting  the  state  at  any  time  be  the  ordered  ages  of  the  cus¬ 
tomers  in  service  at  that  time.  That  is,  the  state  will  be  jc  ■ 
(x^,X2>  ....  xn)  ,  x^  £  X2  <,  . . .  £  x  ,  if  there  are  n  custom¬ 
ers  in  service"  the  most  recent  one  Raving  arrived  x..  time  units 
ago,  the  next  most  recent  arrival  being  x_  time  units  ago,  and 
so  on.  The  process  of  successive  states  will  be  a  Markov  process 
in  Che  sense  that  the  conditional  distribution  of  any  future  state 
given  the  present  and  all  the  past  states  will  depend  only  on  the 
present  state.  In  addition,  let  us  denote  by  X(t)  ■  g(t)/G(t) 
the  hazard  rate  function  of  the  service  distribution. 

We  will  attempt  to  use  the  reverse  process  to  obtain  the 
limiting  probability  density  ptx^.x-,  ....  x)  ,  1  <  a  <  k  , 
x  <_  X2  £  • . .  i.  xn  >  and  P(<j>)  the  limiting  probability  that 
cne  system  is  empty.  Mow  since  the  age  of  a  customer  in  service 
increases  linearly  from  0  upon  its  arrival  to  its  service  time 
upon  its  departure,  it  is  clear  that  if  we  look  backwards,  we 
will  be  following  the  excess  or  additional  service  time  of  a 
customer.  As  there  will  never  be  more  than  k  in  the  system, 
we  make  the  following  conjecture. 

Conjecture.  In  steady  state,  the  reverse  process  is  also  a 
k  server  loss  system  with  service  distribution  G  in  which 
arrivals  occur  according  to  a  Poisson  process  with  rate  X  . 

The  state  at  any  time  represents  the  ordered  residual  service 
times  of  customers  presently  in  service.  In  addition,  the 
limiting  probability  density  is 


n 

xn  n  G(xi) 

p(*i . V  — : —  •  *i  ±x„ 

Z  («[S]>  /l! 

i-0 


and 


P<*) 


l  (XE[S]> i/i! 

U-o 


To  verify  the  conjecture,  for  any  state  x  ■  (x^,  ...,  x^,  ...,  xr) , 
let  e^x)  -  (x^,  ....  xn)  •  N°w  in  the  original 

process  when  the  state  is  x  ,  it  will  instantaneously  go  to 
e^x)  with  a  probability  density  equal  to  A(x^)  since  the 


4 


person  whose  time  in  service  is  x^  would  have  to  instantaneous¬ 
ly  complete  its  service.  Similarly  in  the  reversed  process  if 
the  state  is  e^x)  ,  then  it  will  instantaneously  go  to  x  if 
a  customer  having  service  time  x^  instantaneously  arrives. 

So  we  see  that 

in  forward:  x  ■*  e^Cx)  with  probability  intensity  A(x.) 
in  reverse:  e.(x)  -*■  x  with  (joint)  probability  intensity 
Ag(xi)  . 

Hence  if  p(x)  represents  the  limiting  density,  then  we  would 
need  that 

p(x)X(xt)  -  p(ei(x))Xg(xi) 
or,  since  \(xi)  -  gCx-J/GCx^  , 
p(x)  -  p(ei(x))AG(xi) 

which  is  easily  seen  to  be  satisfied  by  the  conjectured  p(x)  . 

To  complete  our  proof  of  the  conjecture,  we  must  consider 

transitions  of  the  forward  process  from  jc  to  (0,x)  • 

(O.Xt.x-,  ...,  x  )  when  n  <  k  .  Now 
i  z  n 

in  forward:  x  -*■  (0,x)  with  instantaneous  intensity  A 
in  reverse:  (0,x)  •»  X  with  probability  1  . 

Hence  we  must  verify  that 

P(x)A  »  p(0,x) 

which  easily  follows  since  G(0)  *  1  . 

Hence  we  see  that  the  conjecture  is  true  and  so,  upon 
integration,  we  obtain 

P{n  in  the  system} 


-  P(4>)An 


//•/ 


X, <X- . . . <x 

1—  z  —  n 


n 

II  G(x  )dx,  ax 
i-1 


dx 

n 


■P<,)KT  /  /  •••/  j  • 

X1,X2* ' * ■ ,Xn 

v 

-  P(*)(XE[S])n/n!  ,  n  -  1,2,  ....  k 


. .  dx 

n 
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where  E(S]  ■  J  G(x) dx  is  Che  mean  service  time.  Also,  we  see 
that  Che  condicional  discribucion  of  Che  ordered  ages  given  chac 
chere  are  n  in  Che  syscem  is 

p{x  |  n  in  Che  syscem}  -  p(x)/P{n  in  Che  syscem} 

n 

-  n!  n  (G(x  ) /E [ S 1 )  . 
i-1 

As  G(x) / E [ S ]  is  jusc  Che  densicy  of  G  ,  che  equilibrium 
discribucion  of  G  ,  chis  compleces  Che  proof. || 

In  addicion,  by  looking  ac  che  reversed  process,  we  also 
have  che  following  corollary. 

Corollary  1.  In  che  Erlang  loss  model,  che  deparCure 
process  (including  boch  cus comers  complecing  service  and  chose 
chac  are  lose)  is  a  Poisson  process  ac  race  X  . 

Proof.  The  above  follows  since  in  che  reversed  process 
arrivals  of  all  cuscomers  (including  chose  chac  are  lose) 
conscicuees  a  Poisson  process. 


2.  THE  SHARED  PROCESSOR  SYSTEM 

Suppose  chac  cuscomers  arrive  in  accordance  wich  a  Poisson 
process  having  race  X  .  Each  cus comer  requires  a  random  amounc 
of  work,  discribuced  according  co  G  .  The  server  can  process 
work  ac  a  race  of  one  unic  of  work  per  unic  eime,  and  divides 
his  cime  equally  among  all  of  che  cuscomers  presencly  in  che 
syscem.  Thac  is,  whenever  chere  are  n  cuscomers  in  che  syscem, 
each  will  receive  service  work  ac  a  race  of  1/n  per  unic  Cime. 

Lee  X(.e)  denoce  che  failure  race  funcCion  of  che  service 
discribucion,  and  suppose  chac  XE[S]  <  1  where  E[S]  is  che 
mean  of  G  . 

To  analyze  che  above,  lec  Che  scace  ac  any  cime  be  che 
ordered  veccor  of  che  amouncs  of  work  already  performed  on 
cuscomers  seill  in  che  syscem.  Thac  is,  che  scace  is  x  ■ 

(x^,X2»  ...,  x^)  ,  x^  <_  x^  <_  . . .  <_  x  if  chere  are  n  cuscomers 
in  che  syscem  and  x.. ,  . . . ,  x  is  che  amounc  of  work  performed 
on  chese  n  cuscomers.  Lec  p(x)  and  P(4>)  denoce  che 
limicing  probabilicy  densicy  and  che  limicing  probabilicy  chac 
che  syscem  is  empey.  We  make  Che  following  conjecCure  regarding 
che  reverse  process. 

ConjecCure.  The  reverse  process  is  a  syscem  of  che  same 
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type,  with  customers  arriving  at  a  Poisson  rate  X  ,  having 
workloads  distributed  according  to  G  and  with  the  state 
representing  the  ordered  residual  workloads  of  customers 
presently  in  the  system. 

To  verify  the  above  conjecture  and  at  the  same  time  obtain 
the  limiting  distribution  let  e .  (x)  ■  (x^, 

x^)  when  x  *  (x^,  ...»  xn)  ,  x^  —  x2  —  ' *  *  —  xn  •  Note  that 

l(xi> 

in  forward:  x  ■*  e^(x)  with  probability  intensity  - 

in  reverse:  e .  (x)  -*■  x  with  (joint)  probability  intensity 
Xd’(x.)  7 

The  above  follows  as  in  the  previous  section  with  the  exception 
that  if  there  are  n  in  the  system  then  a  customer  who  already 
had  the  amount  of  work  x.  performed  on  it  will  instantaneously 
complete  service  with  probability  Xtx^/n  . 

Hence,  if  p(x)  is  the  limiting  density  then  we  need  that 

A(x  ) 

p(x)  — - —  “  p(ei(x))XG’ (xt) 
or,  equivalently, 

p(x)  *  nG(xi)p(ei(x))X 

-  nG(xt)(n  -  l)G(Xj)p(ej (x))A2  ,  i  #  i 

.  n 

=■  nix"?^)  H  G(x . )  .  (1) 

i-1  1 

Integrating  over  all  vectors  x  yields 
P{n  in  system}  *  (XE[S])nP(<?)  . 

Using 

CO 

P (d>)  +  £  P{n  in  the  system}  *  1 

n*l 

gives 

P{n  in  the  system}  *  (XE[S])n(l  -  XE[S])  ,  n  >_  0  . 

Also,  the  conditional  distribution  of  the  ordered  amounts  of 


work  already  performed,  given  n  in  the  system  is,  from  (1) 


p(x  |  n)  *  p(x)/P{n  in  system; 
n 

-  n!  FI  (G(x  )/E(S})  . 
i-1 

That  is,  given  n  customers  in  the  system  the  unordered  amount 
of  work  already  performed  are  distributed  independently  according 
to  Gg  ,  the  equilibrium  distribution  of  G  . 

All  of  the  above  is  based  on  the  assumption  that  the  con¬ 
jecture  is  valid.  To  complete  the  proof  of  its  validity,  we 
must  verify  chat 

p(x)A  *  p(0,x)  . 

The  above  being  the  relevant  equation  since  the  reverse  process 
when  in  state  (e,x)  will  go  to  state  x  in  time  (n  +  l)e  . 

As  the  above  is  easily  verified,  we  have  thus  shown 

Theorem  2.  For  the  Processor  Sharing  Model,  the  number  of 
customers  in  the  system  has  the  distribution 

P{n  in  system}  •  (*E[S])n(l  -  XE[S])  ,  n  >_  0  . 

Given  n  in  the  system,  the  completed  (or  residual)  workloads 
are  independent  and  have  distribution  G  .  The  departure 
process  is  a  Poisson  process  with  rate  ?  . 

If  we  let  L  denote  the  average  number  in  Che  system,  and 
W  ,  the  average  time  a  customer  spends  in  the  system  then 


L  -  l  n(XE[S])n(l  -  \E[  S] ) 
n“0 


,  AE [S 1 
1  -  XE[S]  ' 


We  can  obtain  W  from  the  well-known  formula  L  «  XW  and  so 


W 


L/X 


.  EI.SJ _ 

1  -  XE[S]  ‘ 


Another  interesting  computation  in  this  model  is  that  of 
Che  conditional  mean  time  an  arrival  spends  in  the  system  given 
its  workload  is  v  .  To  compute  this  quantity,  fix  y  and  say 
that  a  customer  is  "special"  if  its  workload  is  between  y  and 
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y  +  £  .  By  L  ■  AW  ,  we  thus  have  that 

Average  Number  of  Special  Customers  in  the  System 

Average  Arrival  Elate  of  Special  Customer  x  Average 
Time  a  Special  Customer  Spends  in  the  System. 

To  determine  the  average  number  of  special  customers  in  the 
system,  let  us  first  determine  the  density  of  the  total  workload 
of  an  arbitrary  customer  presently  in  the  system.  Suppose  such 
a  customer  has  already  received  the  amount  of  work  x  .  Then 
the  conditional  density  of  its  workload  is 

f(w  |  has  received  x)  *  g(w)/G(x)  ,  x  <_  w  . 

But,  from  Theorem  2,  the  amount  of  work  an  arbitrary  customer  in 
the  system  has  already  received  has  the  distribution  G  .  Hence 
the  density  of  the  total  workload  of  someone  present  in  the 
system  is 

w 

f(w)  - 

“/f[s}dx  ’  Since  dGeU)  *  C(x)/E[S] 

0 

■  wg(w)/E[S]  . 

Hence  the  average  number  of  special  customers  in  the  system  is 
E[number  in  system  having  workload  between  y  and  v  +  e] 

*  Lf (y) £  +  o  (e) 

■  Lyg(y) e/E[S]  +  o(e)  . 

In  addition,  the  average  arrival  rate  of  customers  whose  workload 
is  between  y  and  y  +  £  is 

Average  arrival  rate  *  \g(y)c  +  o(e)  . 

Hence  we  see  chat 


f  dG  (x) 

i  =<*>  “ 


E[time  in  system  \  workload  in  (y  ,  y  +  e) ] 

,  lyg(y) t  .  o(e) 

E(S]Xg(y)e  e 


f 
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LeCting  e  —  0  ,  we  obtain 

E(time  in  system  [  workload  is 


L 


.  y _ 

1  -  \E [S  ]  ' 


Thus  the  average  time  in  the  system  of  a  customer  needing 
y  units  of  work  also  depends  on  the  service  distribution  only 
through  its  mean. 


3.  THE  G/M/k  QUEUE 

In  this  model  we  suppose  that  there  are  k  servers,  each  of 
whom  serves  at  an  exponential  rate  u  .  We  allow  the  time 
between  successive  arrivals  to  have  an  arbitrary  distribution  G  . 
In  order  to  ensure  that  a  steady-state  (or  limiting)  distribution 
exists,  we  assume  1/u^,  <  ku  where  is  the  mean  of  G  . 

To  analyze  this  model,  we  will  use  an  embedded  Markov  chain 
approach.  Define  X  as  the  number  in  the  system  as  seen  by 
the  nth  arrival.  Then  it  is  easy  to  see  that  {X  ,  n  >_  0}  is 
a  Markov  chain.  n 


To  derive  the  transition  probabilities  of  the  Markov  chain, 
it  helps  to  note  the  relationship 


X 


n+1 


X  +  1  -  Y  ,  n  >  0 
n  n  ’  — 


where  denotes  the  number  of  departures  during  the  inter¬ 

arrival  time  between  the  nth  and  (n  +  l)st  arrival.  The 
transition  probabilities  can  be  calculated  as 


Case  (i):  1  >  i  +  1  .  In  this  case.  P..  *  0  . 

-  ij 

Case  (ii)  :  j  <_  i  +  1  <_  k  .  In  this  case, 

P.  .  =  P { i  +  1  -  j  of  i  +  1}  services  are  completed  in  an 
interarrival  time 


CO 

J*P{i+l-j  of  i+1  are  completed  j  interarrival 
0 

time  is  t}dG(t) 


00 


0 


-ut 

e 


) 


i+l-j 


(e 


-ut 


j 

)  dC(t)  . 
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Case  (iii):  i  +  1  >_  j  ^  k  .  To  evaluate  P .  .  ,  in  this 
case,  we  first  note  that  when  all  servers  are  busy^the  departure 
process  is  a  Poisson  process  with  rate  ky  .  Hence, 

P{i  +  1  -  j  departures  in  time  t}dG(t) 

i+l-j 


ij 


00 

-J 


I*  -kyt  7-7 
J  e  (i 


(kyt) ' 


i+l-j) 


7  dG( t)  . 


Case  (iv):  i  +  1  >_  k  >  j  .  Conditioning  first  on  the 
interarrival  time  and  then  on  the  time  until  there  are  only  k 
in  the  system  (call  this  latter  random  variable  T^)  yields 


oo 

J 


Pjj  ■  I  P{i  +  1  -  j  departures  in  time  t)dG(t) 


OO  t 

J*J*P{i  +  l-  j  departures  in  t  |  •  s} 

0  0 


,  -kys  (kus)i  k  .  , 

kue  (i  -  kj'V  dsdG^c) 


"  t 

J/U) 

0  0 


(1  -  e-u(t-s))k"J(e-y(t-s))j 


i-k 


,  -kys  (kus)  .  , 

kwe  ^ ,  dsdG(t)  , 

We  now  can  verify  by  a  direct  substitution  into  the  equa¬ 
tions  -  £  ^i^ij  that  the  limiting  probabilities  of  this 

Markov  chain  are  of  the  form 

*k-l+j  m  *  0,1»  • • •  • 


Substitution  into  any  of  the  equations  £  n^P^  when  j  >  k 

yields  that  S  is  given  as  the  solution  of  i 
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8  «■  J  e'kut(1'S)dG(t)  . 
0 


The  values  ....  ir^-  ,  can  be  obtained  by  recursively 

solving  the  first  k  -  1  of  the  steady-state  equations,  and  c 

x 

can  then  be  computed  by  using  £  tt  .  =■  1  . 

0  1 


If  we  let  W*  denote  the  amount  of  time  that  a  customer 
spends  in  queue,  then  we  can  show,  upon  conditioning,  that 


W* 


Exp  (ky  (1  -  S)) 


k-1 

with  probabilitv  £  it .  *  1 

0  1 


with  probability 


where  Exp  (ku(l  -  3))  is  an  exponential  random  variable  with 
rate  ky(l  -  6)  . 


4.  THE  FINITE  CAPACITY  M/G/k 

In  this  section,  we  consider  an  M/G/k  queuing  model  having 
finite  capacity  N  .  That  is,  a  model  in  which  customers, 
arriving  in  accordance  with  a  Poisson  process  having  rate  \  , 
enter  the  system  if  there  are  less  than  N  others  present  when 
they  arrive,  and  are  then  serviced  by  one  of  k  servers,  each 
of  whom  has  service  distribution  G  .  Upon  entering,  a  customer 
will  either  immediately  enter  service  if  at  least  one  server  is 
free  or  else  join  the  queue  if  all  servers  are  busy. 

Our  objective  is  to  obtain  an  approximation  for  W  ,  the 
average  time  an  entering  customer  spends  waiting  in  queue.  To 
get  started  we  will  make  use  of  the  idea  that  if  a  (possibly 
fictitious)  cost  structure  is  imposed,  so  that  entering  customers 
are  forced  to  pay  money  (according  to  some  rule)  to  the  system, 
then  the  following  identity  holds — namely, 

time  average  rate  at  which  the  system  earns  ■  average 
arrival  rate  of  entering  customers  x  average  amount 
paid  by  an  entering  customer. 

By  choosing  appropriate  cost  rules,  many  useful  formulae 
can  be  obtained  as  special  cases.  For  instance,  by  supposing 
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chat  each  customer  pays  $1  per  unit  time  while 


in  service, 


yields 


average  number  in  service  *»  \(1  -  P  )E[S]  . 

Also,  if  we  suppose  that  each  customer  in  the  system  pays  $x 
per  unit  time  whenever  its  remaining  service  time  is  x  ,  then 
we  get 


V 


(S  -  x)dx 


\(1  -  Pn)[E[S]Wq  +  E { S 2 ] / 2 ] 


where  V  is  the  (time)  average  amount  of  work  in  the  system  and 
where  W*  is  a  random  variable  representing  the  (limiting) 
amount  or  time  that  the  nth  entering  customer  spends  waiting  in 
queue . 


The  above  gives  us  one  equation  relating  V  and  W  and 
one  approach  to  obtaining  W  would  be  to  derive  a  second  equa¬ 
tion.  An  approximate  second^equation  was  given  by  Nozaki-Ross 
in  [3]  by  means  of  the  following  approximation  assumption. 


Approximation  Assumption 


Given  that  a  customer  arrives  to  find  i  busy  servers, 
i  >  0  ,  at  the  time  he  enters  service  the  remaining  service  times 
of  the  other  customers  being  served  are  approximately  independent 
each  having  the  equilibrium  service  distribution. 


Using  the  above  assumption  as  if  it  was  exactly  true,  Nozaki- 
Ross  were  able  to  derive  a  second  relationship  between  V  and 
W  which  resulted  in  an  expression  for  W  as  a  function  of  P^  . 
By  approximating  P^  by  its  known  value  in  the  case  where  the 
service  distribution  is  exponential,  Nozaki-Ross  came  up  with  the 
following  approximation  for  . 


W„ 


mil  Y  (\E.c.sj)_J  _  .  _ 

2E(S]  i  ,  ,,  j-k  (N  10 
j-k  k!kJ _ 


V  (AE[S])j  N;X  (AE[S])j 
L  it  'p  L  4_v 
j-0  j*k  k!kJ  K 


E[Sl(AEfSl)N 


(k  -  AE[S]) 


The  idea  of  an  approximation  assumption  to  approximate 
various  quantities  of  interest  of  Che  model  M/G/k  was  also  used 
by  Tijms,  Van  Hoorn,  and  Federgruen  [6].  They  used  a  slightly 
different  approximation  assumption  to  obtain  approximations  for 
the  steady  state  probabilities.  Other  approximations  for  the 
M/G/k  are  also  given  in  Boxaa,  Cohen,  and  Huffels  [1],  and 
Takahashi  [5]. 
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